Import necessary libraries for the experiment

Number of records in the train and test data

CORRELATION BETWEEN DATA LABELS

The correlation figure below shows that Toxic" comments are clearly correlated with both "obscene" and "insult" comments. Interestingly, "toxic" and "severe_toxic" are only weakly correlated. While we can also observe that, "Obscene" comments and "insult" comments are also highly correlated, which makes perfect sense.

DATA PREPROCESSING

Deep Neural Networks input layers make use of input variables to feed the network for training the model. But in this task (experiment), we're dealing with words text. How do we represent these words in order to feed our model?

In our experiment, we used densed representation of those text (comments) and their semanticity together. The advantage of using this approach is the best way for fitting neural networks onto a text data (as in our case), as well as less memory usage compared to other sparse representation approaches.

Word Embedding

Two ways to feed embeddings to neural networks:

For the first embedding, we used keras preprocessing (Text Preprocessing) libraries. This class allows to vectorize a text corpus, by turning each text into either a sequence of integers (each integer being the index of a token in a dictionary) or into a vector where the coefficient for each token could be binary, based on word count, based on tf-idf

We use cross validation to split arrays or matrices of train data into random train and validation subsets

METHODS ( CNN, RNN, LSTM, all using Glove)

We used early callback functionality that allows you to specify the performance measure to monitor, the trigger, and once triggered. It will stop the training process.

Using Precision, Recall, F1-Measure, AUC, mean etc evaluaiton metrics to evaluate our model

A) "Word2Vec" as a pre-trained Embedding

Prepare the pre-trained embedding layer

Prepare the embedding matrix vectors in order to feed/pass the neural network

Visualize the Orchestrated CNN-Glove Model Architecture

Plot Training & Validation Accuracy with the Loss values of the CNN-Glove Model

Visualize the Orchestrated RNN-Glove MODEL Architecture

Plot Training & Validation Accuracy with the Loss values of the RNN-Glove Model

3) LSTM with Glove

Plot Training & Validation Accuracy with the Loss values of the LSTM-Glove Model

4) Bidirecitional LSTM with Glove

Visualize the Orchestrated Bidirectional LSTM-Glove Model Architecture

Plot Training & Validation Accuracy with the Loss values of the Bidirectional LSTM-Glove Model

Visualize the Orchestrated GRU+Glove MODEL Architecture

Evaluate the GRU+Glove Model train and test score

Plot Training & Validation Accuracy with the Loss values of the GRU+ Glove Model

Plot Training & Validation Accuracy with the Loss values of the Bil GRU + Glove Model

Evaluation of Experiment With GLOVE Pre-trained Embedding

Computing the highest of the evaluation matrics (per model)

Computing the mean of the evaluation matrics (per model)

Score Confusion Table